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Abstract —One practical open problem is the development of a 
distributed algorithm that achieves near-optimal utility using only 
a finite (and small) buffer size for queues in a stochastic network. 
This paper studies utility maximization (or cost minimization) in 
a finite-buffer regime and considers the corresponding delay and 
reliability (or rate of packet drops) tradeoff. A floating-queue 
algorithm allows the stochastic network optimization framework 
to be implemented with finite buffers at the cost of packet 
drops. Further, the buffer size requirement is significantly smaller 
than previous works in this area. With a finite buffer size of 
B packets, the proposed algorithm achieves within 0(e~ B ) of 
the optimal utility while maintaining average per-hop delay of 
O(B) and an average per-hop drop rate of 0(e~ B ) in steady 
state. From an implementation perspective, the floating-queue 
algorithm requires little modification of the well-known Drift- 
Plus-Penalty policy (including MaxWeight and Backpressure 
policies). As a result, the floating-queue algorithm inherits the 
distributed and low complexity nature of these policies. 

I. Introduction 

Stochastic network optimization is a general framework for 
solving a network optimization problem with randomness |T) . 
The framework generates a control algorithm that achieves 
a specified objective, such as minimizing power cost or 
maximizing throughput utility. It is assumed that the network 
has random states that evolve over discrete time. Every time 
slot, a network controller observes the current network state 
and makes a control decision. The network state and control 
decision together incur some cost and, at the same time, serves 
some amount of traffic from network queues. The algorithm is 
designed to greedily minimize a drift-plus-penalty expression 
every slot. This greedy procedure is known to minimize time 
average network cost subject to queue stability. 

This general framework has been used to solve several 
network optimization problems such as network routing 0, 
throughput maximization El, dynamic power allocation a, 
quality of information maximization Q. The framework yields 
low-complexity algorithms which do not require any statistical 
knowledge of the network states. Therefore, these algorithms 
are easy to implement and are robust to environment changes. 
Further, they achieve an [0(1/U), 0{V)\ utility-delay trade¬ 
off, where V > 0 is a parameter that can be chosen as desired 
to achieve a specific operating point on the [0(1/V) 1 0(V)\ 
utility-delay tradeoff curve. 
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Prior works attempt to improve network delay without 
sacrificing reliability, where reliability is measured by the rate 
of packet drops. Previous works El and El use an exponential 
Lyapunov function and assumed knowledge of an e parameter, 
where e measures a distance associated with the optimal op¬ 
eration point. They achieve an optimal [0(1/U), 0(log(U))] 
utility-delay tradeoff. A simpler methodology allows packet 
drops in order to obtain an [0{1/V), 0([log(U)] 2 )] utility- 
delay tradeoff El, GO- In GQ, a steady state behavior is ob¬ 
served to learn a placeholder parameter to achieve the tradeoff 
in steady state. However, the algorithm does not gracefully 
adapt to changes of the network state distribution. It would 
need another mechanism to sense changes and then recompute 
a new placeholder parameter with each change. The Last- 
In-First-Out (LIFO) queue discipline is employed to resolve 
this issue 0. However, these works, which achieve average 
queue size that grows logarithmically in V, still assume the 
availability of infinite buffer space El-El. 

A practical implementation of the LIFO scheme is devel¬ 
oped in ED. The work in ED also introduces a floating- 
queue algorithm, operating under the LIFO scheme, to deal 
with finite buffers. The algorithm in flOl is heuristic, and it 
is not clear how to analyze its behavior. The current work is 
inspired by this floating queue idea of IflOl and adopts the 
same “floating queue” terminology, even though the floating- 
queue algorithm developed here is different from IflOl . Indeed, 
the floating queue technique of this paper operates under the 
First-In-First-Out (FIFO) scheme. It splits each queue into two 
queues (one for real and one for fake packets) and yields 
analytical guarantees on utility, delay, and packet drops. 

Several backpressure approaches GD-ED attempt to im¬ 
prove network delay. However, those focus on specific as¬ 
pects and do not have the theoretical utility-delay tradeoff. 
Stochastic network optimization with finite buffers has been 
studied previously in M- That work uses a non-standard 
Lyapunov function and knowledge of an e parameter to derive 
an upper bound on the required buffer size. However, the e 
parameter can be difficult to determine in practice, and the 
resulting utility-delay tradeoff is still [0{\/V),0(V)\. An 
implementation of that work is studied in fT31 . 

This paper develops a floating-queue approach to general 
stochastic network optimization with finite buffers. Our algo¬ 
rithm is inspired by finite buffer heuristics in IflOl and the 


steady state analysis in HQ- We propose the floating-queue 
algorithm to solve the learning issue in j8|. The result obtains 
the best of both worlds: It achieves the desired steady state 
performance but is just as adaptive to network changes as 
LIFO scheduling. For finite buffers of size B, deviation from 
utility optimality is shown to decrease like 0(e~ B ) and packet 
drops are shown to have rate 0(e~ B ), while average per-hop 
delay is 0(B). 

This paper is organized as follows. The system model is 
described in Section hd Section m describes the standard 
drift-plus-penalty approach. The floating queue algorithm is 
introduced in Section [IV] Performance of the floating-queue 
algorithm is analyzed in Section [V] and is validated by simu¬ 
lation in Section [VlJ Section 1 VIII concludes the paper. 

II. System Model 

The network model of this paper is similar to that of (8j. 
Consider a network with N queues that evolve in discrete 
(slotted) time t £ {0,1,2,...}. At each time slot, a network 
controller observes the current network state before making 
a decision. The goal of the controller is to minimize a time 
average cost subject to network stability. An example of time 
average cost is average power incurred over the network. 
Utility maximization can be treated by defining the slot-/ cost 
as —1 times a slot-/ reward. An example utility maximization 
problem is to maximize time average network throughput. 
The rest of the paper deals with cost minimization, with the 
understanding that this can also treat utility maximization. 

A. Network State 

The network experiences randomness every time slot. This 
randomness is called the network state and can represent 
a vector of channel conditions and/or random arrivals for 
slot t. Assume there are M different network states. Define 
S = {si, S 2 , • • •, Sm} as the set of all possible states. Let 
5(f) denote the network state experienced by the network at 
time t. Let n m £ [0,1] be the steady state probability that 
S(t) = s m , i.e., 7r m = P{5(f) = s m }. For simplicity, it is 
assumed that 5(f) is independent and identically distributed 
(i.i.d.) over slots. The same results can be shown in the general 
case of ergodic but non-i.i.d. processes (see |{8)). The network 
controller can observe 5(f) before making the slot-f decision, 
but the 7 r m probabilities are not necessarily known to the 
controller. 



Fig. 1. Arrivals and services at a standard queue 


2) Queues are served. The service variables are fiijit), 
representing the integer amount of packets taken from queue i 
and transmitted to queue j, for all i , j £ Af ={1,..., N}. This 
is determined by a function fiij(t)=fiij(x(t), 5(f)) : X s A —»• 
Z+. Further, the decision admits fioi(t)=noi(x(t), S(t)) : 
X sl ' t ' 1 —» Z + integer amount of exogenous packets to queue 
i £ Af. Packets depart from the network at queue j £ Af with 
an integer amount /Xjo(f)— Hjo(x(t), 5(f)) : X s A —t Z+. 
Note that we set jMi(t) = 0 for all z £ Af U {0} and for 
all t. The transmission, admission, and departure are shown in 
Figure Q] 

For every s m £ S , we assume functions /(-,s m ) and 
IMji/i s rn ) for i,j £ Af LI {0} are time-invariant, and mag¬ 
nitudes of Y2iLoh ‘"(Am) and Ejlo Fnj(-, s m ) are upper 
bounded by constant S( max > £ (0, oo) for every n £ Af. 
Furthermore, the network optimization is assumed to satisfy 
the following Slater condition j8): Let \X\ be the cardinality 
of set X. For every s m £ S and k £ {T,..., |/f Sm |}, there 
exist probabilities (such that Y^k= t ' Cfc m = 1 for all 
s m £ S ) that define a stationary and randomized algorithm. 
Whenever the network controller observes state 5(f) = s m , 
the stationary and randomized algorithm chooses action x s k m 
with conditional probability . The Slater condition assumes 
there exists such a stationary and randomized algorithm that 
satisfies: 


m \x 3m \ 

m=l k—1 


N N 

) s m ) ~ ^ , S m ) 

3=0 

< —rj for all n £ Af , 


i=0 


for some ?/ > 0. In fact, this assumption is the standard Stater 
condition of convex optimization C6|. 


B. Control Decision 

Every time slot, the network controller chooses a decision 
from a set of feasible actions which depends on the current 
network state. Formally, define X s A) as q le decision set 
depending on 5(f), and let x(f) denote the decision chosen by 
the controller at time f, where x{t) £ X s< d\ Assume the action 
set X Sm is finite for every s m £ S. On slot f, these control 
decision and network state (x(f),5(f)) affects the network in 
2 aspects: 

1) A cost is incurred. The cost is /(f)—/(x(f), 5(f)) : 
X s A) m An example cost is energy expenditure. Another 
example is —1 times the amount of newly admitted packets. 


C. Standard Queue 

The network consists of N standard queues. Let Q n (t) 
denote the backlog in queue n at time f, and let 
Q(f)=(Qi(f),...,Q JV (f)) be the vector of these backlogs. 
The backlog dynamic of queue n £ N} is 


Qn{t+l) = max 


N 

\ ' 


Qn(t) - 2^Mnj(f),0 

3=0 


N 

i=0 


,(t). (i) 


When there are not enough packets in a queue, i.e, Q n (t) < 
o d’nj (f). blank packets are used to fill up transmissions. 






















Initialization: Q(0) = 0 
for t G {0,1,2,...} do 
Observer S(t) and Q(t). 

Choose x(t) that solves <[3]). 

Update Q n (t + 1) according to © Vn G A f. 
end for 


Fig. 2. The drift-plus-penalty algorithm 


cost, which can be made as small as desired by choosing 
a sufficiently large value of V. The equality © implies 
that average queue backlog grows linearly with V. Applying 
Little’s law gives the [0(1/V), 0(V)\ utility-delay tradeoff 
(see El for a standard description of Little’s law). 

Notice that the drift-plus-penalty algorithm assumes infinite 
buffer size at each queue, even though the average queue size 
is bounded by 0(V). 


T-l N 

limsup- ^ ^E[Q n (f)] 

t-voq i ; —: 


D. Stochastic Formulation 

The controller seeks to minimize the expected time-average 
cost while maintaing queue stability. The expected time aver¬ 
age cost is defined by 

1 T_1 

/ = limsup-^E[/(f)], 

T_HX) J t=0 

and the queue stability is satisfied when 

T—l N 

~ l ,] < oo. 

t=0 n= 1 

The stochastic network optimization problem is 

Minimize / (2) 

Subject to queue stability. 

III. Drift-Plus-Penalty Method 
A. Drift-Plus-Penalty Method 

The drift-plus-penalty method of © can solve problem © 
via a greedy decision at each time slot that does not require 
knowledge of the steady state probabilities. The method has 
parameter V > 0. In the special case of V = 0, this policy is 
also called “MaxWeight” or “Backpressure”: 

Drift-Plus-Penalty Policy: At every time t G {0,1,2,... }, 
the network controller observes network state S(t) and back¬ 
log vector Q(t). Decision x(t) G X Sm is chosen to solve: 

Minimize Vf(x(t), S(t)) (3) 


N 

E 

n=1 


Qn(t) 


N N 

^ p, in (x(t),S{t)) - y ^p n j(x(t),S(t)) 


i =0 


3=0 


Subject to x(t) G 

Depending on the separability structure of problem ©, it 
can be decomposed to smaller subproblems that can be solved 
distributively. The algorithm is summarized in Figure [2] 

It has been shown in © that 

/(DPP) </(opt)+0(^1/) (4) 


T-l N 


lim sup — EE E [Q n (t)] = 0(V), 


T—> oo 


(5) 


t=0 n= 1 


B. Deterministic Problem 

In order to consider a finite buffer regime, the steady-state 
behavior of the drift-plus-penalty algorithm is considered. In 
GO. the stochastic problem © is shown to have an associated 
deterministic problem as follows: 


M 


( 6 ) 


Minimize V n m f(x Sm ,Sm) 

m=1 

M N 

Subject to E 

(x Sm ,s m ) Vn G M (7) 


m —1 i=0 

M N 

< E ^ ^ l-^nj (s 

m—1 j —0 

x Sm g X s m Vm g {1 


Let 7 = (71,..., 7 jv ) be a vector of dual variables associ¬ 
ated with constraint ©. The dual function of problem © is 
defined as: 
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( 8 ) 


This dual function ® is concave. Therefore, the following 
dual problem is a convex optimization problem: 


Maximize <7(7) 
Subject to 7 G I 


(9) 


1 N 


Let 7 V * = ( 7 ^*,..., yXr *) be a vector of Lagrange mul¬ 
tipliers, which solves the dual problem © with parameter 
V. The following theorem from El describes a steady state 
property of the drift-plus-penalty algorithm: 

Theorem 1: Suppose y v * is unique, the Slater condition 
holds, and the dual function g( 7 ) satisfies: 


g{l*) > 3 ( 7 ) + L 7 K * — 7 for all 7 G 


*+, 


where /( DPP ) is the expected time average cost achieved by 
the drift-plus-penalty policy, and /1 °p 9 is the optimal cost of 
problem ©. The inequality © implies that the drift-plus- 
penalty policy achieves cost within 0 ( 1 /V) of the optimal 


for some constant L > 0, independent of V. Then under 
the drift-plus-penalty policy, there exist constants D,K,c*, 
independent of V, such that for any /3 > 0, the following 
upper bound holds 


V(D, K0) < c*e -/3 , 


( 10 ) 







where 


V{D,Kf3) 

1 T_1 

— limsup — V] P { 3n , | Q n (t) - 7 ^*| > D + K/3}. (11) 

T ^°° 1 t=0 

Proof: Please see the full proof in |[ 8 j. ■ 

As all transmissions, admissions, and departures are inte¬ 
gers, the queue vector has a countably infinite number of 
possibilities, and under a mild ergodic assumption the steady 
state distribution of {Q(t) : t > 0} exists. In this ergodic 
case, the r P(D 1 K[3) value in (fill) becomes the steady state 
probability that backlog deviates more than D+K/3 away from 
the vector of Lagrange multipliers. Note that the probability in 
( fTTTb vanishes exponentially in 3. This implies that, in steady 
state, a large portion of arrivals and services occur when the 
queue backlog vector is close to the Lagrange multiplier vector 
r y v *. Thus, if we can admit and serve this portion of traffic 
using finite-buffer queues, the network still operates near its 
optimal point. 

IV. Floating-Queue Algorithm 

In this section, the floating-queue algorithm is presented 
as a way to implement the drift-plus-penalty algorithm using 
finite buffers. The algorithm preserves the dynamics of the 
drift-plus-penalty algorithm and hence inherits several of its 
performance guarantees. 

Recall that standard queue n £ TV has dynamic ([I]). To 
simplify notation, let a n (t)= E^o Pin(t) denote aggregated 
arrivals to queue n, and let & n (£)=EyLo Pnj(t) denote ag¬ 
gregated services from queue n at time t. This implies that 
j(max) U pp er bounds both a n (t) and b n (t). The dynamic <[TJ 
can be written as 

Q n {t + 1) = max [Q n (t) - b n (t), 0] + a n (t). ( 12 ) 

For the rest of this paper, the above dynamic is considered for a 
standard queue. Note that a n (t) and b n (t) are fully determined 
after knowing all p l: j (t) from the drift-plus-penalty algorithm. 

A. Queue Transformation 

In the floating-queue algorithm, each standard queue n € J\f 
of Section IH-CI is a combination of a real queue and a 
fake queue. The real queue has buffer size B for stor¬ 
ing real packets. The fake queue contains fake packets 
and only requires a counter to implement. Let Q r n [t ) and 
Qh(t) denote respectively the amount of backlogs in the real 
queue and the fake queue of the standard queue n. Define 
Q r (*)=(QI(*). ■••.<&(*)) ^d Qf ..., Q f N (t)) 

as vectors of real and fake queue backlogs. We use the term 
floating queue n to refer to the 2 -queue combination consisting 
of real and fake queues n. 



B. Real and Fake Parts of Arrivals and Sendees 

At each queue n £ A f. let and 6^(f) denote the 

aggregated real and fake serviced packets at time t. the 
floating-queue algorithm always serves real packets before 
fake packets as: 

b r n (t) = min [Q r n {t),b„{t)] (13) 

b l(t) = b n (t) ~b r n (t). (14) 

It is easy to see that b n (t) = b r n (t) + b^ft). Also, b r n {t) and 
&£( t) are fully determined, since b n (t) and Q r n (t) are known. 

To differentiate the real and fake packets in the drift-plus- 
penalty variable Pij(t), let and p{j(t) denote the real 

and fake parts of pij(t) for i,j £ J\f U {0}. All pij(t) are 
non-negative integers. Since pu(t) = 0, we have p\ t (t) = 
Plf t) = 0 for all z £ ff U {0}. All exogenous arrivals are 
considered as real packets, so knowing the value of poj(t), we 
set p r Qj (t) = p 0j (t) and p f oj {t) = 0 for every j £ {1,..., TV}. 

The real and fake parts of Pij(t) for i £ TV, j £ TVU {0} can 

be set arbitrarily to satisfy: 

N N 

b n(t) = Fuji*), b l(t) = ^(t) Vn £ TV 

3 —0 3=0 

Pij 0) = Pij (t) + Pij (t) Vi £ TV, j £ TV U {0} 

Therefore, all pf (t) and p{j(t) for all i, j £ TVu{0} are fully 
determined. This is illustrated in Figure [3] 

Let a r n {t) = E^o Pinit) and a n(t) = EZiPLtt) be 

the aggregated real and fake parts of a n (t). They are fully 
determined and a n (t) = a r n {t) + a£(t). 

The floating-queue algorithm always admits a real packet 
to a real queue as much as allowed by its buffer space. Let 
aj} (t) denote the amount of packets in a r n (t) that are admitted 
to the real queue n at time t. Another part of a r n (t), which is 
dropped, is denoted by d n (t) and becomes fake packets. Let 
al (t ) denote the admitted fake arrivals, including original fake 
arrivals a£(t) and dropped packets d n (t). The arrival dynamic 
of queue n £ TV is 


a r n (f) = min [B - Q r n (t), a r n (t)] 

(15) 

dn(t) = a r n (t) -a r n(t) 

(16) 

a n(t) = +d n (t). 

(17) 


It is easy to see that a^(t) + a^(t) = a r n {t ) + a f n (t). 











C. Real and Fake Queuing Dynamics 

The dynamics of real and fake queues of standard queue 
n G J\T are 

Qn(t + 1 ) = Quit) - K(t) + a n(t) (18) 

Q f n{t+ 1) = max [Qn(t) -blit), 0] + a({t). (19) 

Lemma 1: From any time to, when Q(fo) = Q r (to) + 
Qf(to), it follows that 

Q(t) = Q r (t) + Q f (t), Vt > t 0 - 

Proof: We prove this lemma by induction. For to, 
Q{to) = Q r (to) + Q (to) by the assumption. Suppose 
Q(t ) = Q r (t) + Qf(t) at time t. At queue n G Af, we have 

Qn(t + 1) = max [Qn(t) - b n (t), 0] + a n (t) 

= max [Q r n (t) + Q f n {t) - b r n (t) - b f n {t), 0] 

+ a n(t) + a({t). (20) 

When there are not enough real packets, Q r n (t) < b n (t), 
it follows that Q r n (t ) — b r n (t) = 0 from (fl3l ). Equation (EUt 
becomes 

Qn(t + 1) = max [Qi(t) - b f n (t),0\ + a£(t ) + <'(f) + 0 
= Qi(t + 1) + Q r n {t + 1). 

When there are enough real packets, Q r n (t ) > b n (t), we 
have that fr£(f) = 0 from (IT3l > and ( TPTI i. Equation (f20t becomes 

Qn(t + 1) = Q f n {t) + a£(t) +Q r n (t) - b r n (t) + a r ^{t) 

= QL(t + 1 ) + Qn(t + !)■ 

Thus, Q n (t + 1) = Qn(t) + Q r n (t) for all n G Af. ■ 

The implication of Lemma Q] is that, although the floating- 
queue algorithm implements these real and fake queues in¬ 
stead of the standard queues, the dynamics of Q(t) and 
Q r (t ) + Qf(t) are the same. Hence, when decision x(t) is 
chosen by solving <[3j with Q r (t ) + Q^(t) instead of Q(t), all 
decisions {s(f)}“ 0 under the standard algorithm (in Figure 
0) are identical to the decisions {x(t)}^f 0 under the floating- 
queue algorithm (in Figure 0, given that Q r (0) + Q*( 0) = 
Q(0). Yet, the buffer size of each real queue in the floating- 
queue algorithm is B. Let Q*( 0) € be a predefined 
fake backlogs. The floating-queue algorithm is summarized 
in Figure 0 

We prove a useful lemma of the floating-queue algorithm, 
which will be used in Section IV-BI 

Lemma 2: Under the floating-queue algorithm, when the 
buffer size of the real queue n £ Af is B > 2<5( max ), if d n (t) > 
0, then Ql(t + 1) > Q f n (t). 

Proof: Event d n (t) > 0 implies that a r n (t) > a r n (t) from 
m and a n(t) = B-Qn(t) from (00, so Q r n (t) > B-a r n (t). 
When B > 2we have Q r n {t) > 2 5^-a r n (t) > 5( m3X \ 

and there are enough real packets for all services. Therefore, 
all services take real packets and b r n (t) = b n (t) and bl(t) = 0 
from (fT3T > and (ITdl ). From (fl9] > and (IT7] >. we have 

Q f n(.t + 1) = Ql(t) + a£{t) = Q f n (t) + a f n {t) + d n (t) > Q f n (t). 


Initialization: Q r (0) = 0 and Qf (0) 

for t G {0, 1,2,.. .} do 

Observer S(t) and let Q(t) = Q r (t ) + Qf (t). 

Choose x(t) that solves 0. 

Calculate ( a n (t),b n (t )) V?t G Af. 

Calculate (6^(f), 6^(t)) as (fl3l > and ([10 Vn G Af. 
Adjust (a r n (t), a£ (t)) as (IT5l)—(fl7]> Vn G Af. 

Qn(t + f) = Quit) - b r n(t) + a r n{t) Vn G Af. 

Qn(t + 1 ) = max [Ql(t) - b* n (t), 0] + a f n (t) Vn G Af. 
end for 

Fig. 4. The floating-queue algorithm 


The interpretation of Lemma 0 is that, for any queue n with 
buffer size B > 2<5^ max - ) , if real packets are dropped at time t, 
then the fake backlogs at time t + 1 always increase. 

V. Performance Analysis 

The steady-state performance of the floating-queue algo¬ 
rithm is analyzed by bounding from below the admitted real 
arrivals at each queue n G Af. Define 0„(f)= 

(a r n (t), a s n (t), (t), a({f),b r n {t), b f n (t ), Q r n (t ), Q f n (t)^j 

as a sample path of the arrivals, services, and backlogs of 
queue n that is generated by the floating-queue algorithm at 
time t. For any positive integer T and starting time to, a sample 
path of queue n from to to to + T is denoted by {Q>n(t)Yff^■ 
Note that Q n {t) can be recovered from this sample path as 
Qn{t) = Qn(t) + Q f n(t). 

From sample path Q n (t), the amount of real arrivals are 
a r n (t), and the amount of admitted real arrivals are a r n (t), 
which depend on the floating-queue mechanism (fl~5] >. To lower 
bound this admitted real arrival a r n (t), we construct another 
mechanism, called “lower-bound policy”, that operates over 
the sample path. It has a different rule for counting admitted 
real packets (later defined as a r n (t)), which is part of the 
real arrivals a r n (f). We will show (Lemma 0 that the amount 
of admitted real arrivals under the floating-queue algorithm 
is lower bounded by the amount of admitted real arrivals 
under the lower-bound policy. Using this lower bound the 
performance of the floating-queue algorithm can be analyzed. 


A. Lower-Bound Policy 

In this section, queue n G Af is fixed and the lower-bound 
policy is defined for this queue. For simplicity, assume that 
the buffer size B is even and B > 2^ max f 

Recall that y v * is the Lagrange multiplier of problem 0, 
and i5( inax ) is the upper bound on a n (t) and b n (t). Define 


B A 

u n — 


yl* -B/2 + <5 (max) , "/Z* + B/2 - 6 (max) 


Let a r n (t) denote the number of admitted real packets under 
the lower-bound policy at time t. Given any sample path 0„(t) 
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Fig. 5. Time interval 7~(T) is partitioned into 7 h(T) and 7l(T). 
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having real arrivals a r n {t) and total backlogs Q n (t ), the lower- Fig. 6. Partitioning 7L(T) into intervals of t fc , t~ and t+ 

bound policy counts real packets as 


«n(*) 


<(i) ,<5n(i)GS n 
0 , Qn(t) f. B n 


( 21 ) 


Let d n (t) denote the number of dropped packets under the 
lower-bound policy at time t. It satisfies 

d n (t) = a r n {t)-a r n (t). (22) 


When K(T) > 0, the time interval in interval {tk, ■ ■ ■, t' k } 
for k £ {1,2 ,...,K(T)} can be partitioned into in¬ 

tervals between local minima and local maxima. Define 
f7(i)=arginf rg{t+11+2 } {Q£(t) > Qi(t)} to be the first 

time index after t that the fake queue increases. For k £ 
{1,2,... ,K{T)},j £ {1,2,...}, let 


Notice that a r n (t) and d n (t) are artificial numbers and are not 
real and fake packets in a real system. These values can be 
determined by a r n (t) and Q n (t) of sample path 0 n (i). 

B. Sample Path Analysis 

The goal of this section is to show (Lemma [ 6 j that, for any 
sample path {0n(t)}£l t of queue n £ Af and any positive 
integer T, the admitted real arrivals under the floating-queue 
algorithm with buffer size B is lower bounded by 

to+T-l to+T-1 

Y Y a nW- 5 - 

t=to t=to 

In this section, queue n is fixed and analyzed; however, the 
analysis results hold for every queue n £ Af. 

For any starting time to and any positive integer T, define 
7~(T)={t o,..., to + Tj as a time interval of consideration. It 
can be partitioned into disjoint sets 7 h (T) and Tf(T), which 
are illustrated in Figure [5] where 

MT)±{t £ T{T) : Q{(t) > rt* - B/2} 
%{T)={t £ T{T) : Qf n (t) < j}/* - B/2}. 

Time interval 7L(T) can be partitioned into disjointed 
intervals of time that starts when the fake queue in the sample 
path satisfies Ql(t) < 7 //* —B/2 and ends when it does not. 
This is illustrated in Figure [ 6 ] For k 7 {1,2,...}, let 

t k = arginf tg{t , fc _ i+1) to+T} [Q f n {t) < 7 ^* - B/2} 

t'k = arginf te{tfc+1 ,..., to+ T } {Q f n (t) > In* ~ B / 2 } - !, 

where t' 0 = to — 1 and arginfjg^ B j{C(f)} = B + 1 if 
A > B or C(t) is not satisfied for all t £ {A, ... ,B}. Let 
K(T) = argmax fe>0 {tk < to + T + 1} denote the number of 
intervals {tk ,..., t' k } contained in T(T). 


hj = min 


tk,j = min 


ax ginf t£ { t +^_ i+ i ) { Qn(t) < Q{[t-1) and 
Q f n (t) < Q f n (r) Vr £ {t + l,...,U(t)}},t' k 
arginf tg { t - i+ i ,...,t' k } {Qn(t) > Qn(t + l)},ffc 


where t k0 = tk — 1 and Q^lf 0 — 1) = 00 . Intuitively, 
during {tk, ■ ■., t' k }, 11 ■ is the first time index that the j th 
local minimum is reached, and t k - is the last time index of 
the ;y th local maximum. This is illustrated in Figure [ 6 ] Let 
J{k) = arginf J>0 j denote the number of local 

maxima during {tk, ■ ■ ■, t ' k }. 

For a technical reason (used in Lemma|4|, let T\{T)={{tk — 
1) £ T(T) : k £ {1,..., K(T)}. The following lemmas hold 
for the real arrivals in 7h(T)\Ta(T) and 7L(T) UTa(T). 

Lemma 3: When B > 26(™ x \ given any sample path 
the following relation holds 


Y a n(t) > Y «nW' 

fer H (T)\r*(T) teT H (T)\r A (T) 


Proof: Two cases are examined. 

1) When Q n (t) £ B n for any t £ 7 h(T)\Ta{T), we have 
a n (t) = a n{t), because real queue n has enough buffer space: 

Quit) = Quit) - Qf n (t) 

< (in* + B/2 - S ("“>) - ( 7 r* - B/2) 

< B - (5 {max) . 


The first inequality holds because of Q n (t) £ B n and t £ 
7h(T). For the lower-bound policy, we have a r n {t) = a r n (t), 
because Q n (t) £ B n . So a r n (t) = d r n {f). 

2) When Q n (t) ^ B n for any t £ %(T)\Ta(T), we have 
a r n (t) > a r n (t) = 0, because Q n {t) ^ B n and a r n (t) > 0. 








































These two cases implies the lemma. ■ 

Lemma 4: When B > 2<j( max \ given sample path 
with Qni^o) = 0’ the following holds 

E “n(*)> E 

teT L (T)ur A (T) ter L (T)ur A (T) 


Proof: For k G {1,..., K(T)}, j G {1,. .., J(k)}, 3 
cases are examined. 

1) For real arrivals a r n [t) during t G jffc — 1,..., t^ 1 — 2| 

(if exists), the fake backlogs Quit) is non-increasing by 
the definition of tf 1 . From Lemma [2] the non-increasing 
implies no packet drops and a r n ( t ) = a r n {t) for t G 

|ffc — 1,... ,tf 1 — 2j. Since a r n {t) < a r n (t), it follows 

that Et=t*-i <£(*) > For a special case 

when t± = to, same argument can be used to obtain 

eE: 2 <'(*)>e£e 2 ^(*). 

2) For real arrivals a r n (t) during t G jfj^- — 1,..., t k - 1, it 
can be shown that V k, ° 

-i 

result is proven in Appendix. 

3) For real arrivals 


<(t)> E t E t - ,&;(*)• This 


k 


i + 1; 


t~ — 2 

> b k,j +1 ^ 


ajj(f) during t G 
| (if exists), the fake backlogs 


Qn{t) is non-increasing by the definitions of fE and t k 


Lemma [2] implies that 


'(*)> E 




;w- 


Since Q’„(fo) = 0, the time interval {tk,... ,t' k } for k G 
{1,... ,K(T)} starts with either the first case or the second 
case and ends with the second case. Thus, it is clear that, if 
real arrivals under the lower-bound policy are dominated over 
every subinterval, then they are also dominated over the union 
of these subintervals. ■ 

Lemma 5: When B > 2<j( max \ given sample path 
{©ra(i)}£E with Qn(to ) = 0 and positive integer T, it holds 
that 

to+T— 1 tQ-\-T —1 

E <{t)> E a nW- 

t=to t=to 


Proof: Disjoint time intervals 7h(T — 1)\7a(? 1 — 1) and 
Ti(T — 1) U 7a (T — 1) are the partitions of T(T — 1). Then 
Lemma [3] and Lemma [4] imply the lemma. ■ 

The above lemma is not general, since it requires Quito) = 
0. Now its general version is provided. 

Lemma 6: When B > 2<5( max ), given any sample paths 
{©ra(f)}^ to and positive integer T, it holds for any Q„ito) G 
{0,1,..., B} that 


to-\-T —1 to-\-T —1 

E <(t)> E 

t=to t=to 

Proof: Construct sample path |©„(f) j with 0„(f) = 

{K it), a f n (t), (t), a£ {t), b r n it), b f n it), Q r n it ), Q f n (t )) and 

• t—i < to, 

• Quit) = Quit) for all t G {f 0 , to + 1, • ■ • }, 


• Quit) = Quito) for all t G {t_i,... ,to - 1}, 

. Q r n {t-\) = 0, Quito) = Quito) = E ~<it\ 

• it) =Kit) =b f nit) = 0 for all t G {f_i,..., f 0 - 1}. 
The last two conditions automatically set the values of 

af {t), Qn(f)} ■ This new sample path satisfies 

Lemma 0 Let a' n (t) denote the admitted real arrivals un¬ 
der the lower-bound policy of the new sample path. Since 

EtltJ” 1 «n it) = ESo T_1 < it), ^ follows that 

t,Q — l to~\-T— 1 to~l to+T—l 

E 5 n(*)+ E a nit)= E <it)+ E 5 n0) 

t=t — 1 t=to t=t — 1 t=to 

to - 1 to+T 1- 1 £o — 1 to+T 1- 1 

> E ^nit)+ E “n(*)= E “nW+ E 

t=t_l t=to t=t — 1 t=to 

The first inequality is the application of Lemma [5] and the last 
equality holds, because {Q n (f)}^E of both original and new 
sample paths are identical. 

Therefore, we have 

to+T —1 to~\~T —1 

E a nit)> E b r n it)-B. 

t=to t=to 

The inequality uses the facts that Et="t_i ®n(f) ^ 0 and 

T,\ 0 ^.Xit) = Quito) <B. 1 ■ 


C. Performance of Floating-Queue Algorithm 

1) Average Drops: The average drops at each queue is 
analyzed using the steady state and sample path results. Recall 
that constants D,K,c* are defined in Theorem Q] 

Lemma 7: Suppose B > 2(<5f max ^ + D). In the steady state, 
the average drops at real queue n G Af under the floating- 
queue algorithm is bounded by 


i t 0 +T —1 , v 

L v—A / X — \B / 2 — <5 ^ max ) — D] 

lim - E E[ 4 (t)l < g (max) c*e a -. 

T->-oo T ^' 

t=to 


Proof: We consider queue n G M. Let I {A} be an 
indicator function of statement A such that I {A} = 1 if 
statement A is true; otherwise I {A} = 0. Equation (l2ll 
can be written as ajj(f) = a£(£)I{Q„(t) G B n } = ajj(f) — 
a r n it)l{Q n it) ^ B„}. Then we have that 


, to+T-l 

J. E E K(f)] 

t=t 0 

1 to+T-l 

= - E V[<it)-<iW{Qnit)tB n }} 

t=t 0 

Dividing the result in Lemma [6] by T and taking an 
expectation yields 


1 

T 


to+T—1 

E E 

t=tn 


to+T—1 


it) >- E E KW] 


B 

T‘ 





Combining the above two relations gives 


1 to+T— 1 to~\-T —1 

- E E K'(*)l>r E E K(*)]-r 


t=t 0 


T 

t=t 0 
to+T—1 


E EK(t)I{QnW^5n}]. 


t=to 


It follow from ( 1 1 6| i that 

1 -i 1 73 

- E E [d n (t)]<- 5 ( ™ x) V{Q n (t)t)3 n } + - 

t=to t=to 

Taking limit as T approaches infinity yields 


1 


to-\-T —1 


lim 7p E E K(^)] 

T —yoo 1 z —' 


t=t 0 


< (5 (max ) lim i V P{Q„(t) ^£„}- (23) 

T 1 —Vrsn / z ' 


to+^ — 1 


T^oo T 


t=to 


In steady state. Theorem |T| with /3 = B//2 ^ - — yields 

to+T 1- 1 


r lim - E V{Qn{t)*B n } 


t=to 

to+T —1 


< 


lim sup i E P { 10«(f) - 7n*| > S/2 - <^ max >} 

T-voo 1 t J 


t=t 0 


= V(D , B/2 - 5 (max) - D) < c * e -[s/2-5(”“)-_D]/i<-_ 

Applying the above bound to (f23l) proves the lemma. ■ 
2) Delay: At each queue, the average delay experienced by 
real packets is derived by invoking Little’s law fT71 . Define 

to-\-T—l to-\-T—l 

«n= lim Tf, E E KW]> d„= lim - E E K0)]- 

T-> oo 1 z —' T— >-oo 1 z —' 

t=to t=to 

Lemma 8: Suppose B > 2(<j( max ) + D). In the steady state, 
the average delay at real queue n G J\f under the floating- 
queue algorithm is bounded by 

Per-hop delay < ■ 

Proof: Since the buffer size of queue n G AT is f?, Little’s 
law implies: 

to+T-l 


Per-hop Delay = B 


lim 1 E E 

-Vno / z -' 


r^oo T 


t=to 


/ i ^ 0+^—1 

= S/ lim - E E KW-il 

' L i=*o 


< 5 


-r _ (5 (max) c * e -[B/2-5(”‘“)-D]/if 


The implication of Lemma[8]is that, when B is large enough 
such that < 5( max ) c * e — [s/ 2 — anc j t jj e num b er 0 f drops 
at other queues are negligible, a r n is approximately a n , and the 
average delay is 0(B). 


3) Objective Cost: The average objective cost is 
considered in two cases. Let fBQ) f) denote the cost 
under the floating-queue algorithm at time t, and 
/(FQ)4limT_>.oo y St=E _1 E [/^ FQ H0] denote the expected 
time-average cost under the floating-queue algorithm. 
Drop-Independent Cost: 

In this case, packet drops do not affect the objective cost. Such 
cost can be the energy expenditure that is spent to transmit 
both real and fake packets. Due to this independence, the 
average cost follows immediately from the result of drift-plus- 
penalty policy (01). 

Theorem 2: Suppose each real queue has buffer size B > 
2(7) (max) + D). When V > 0 and packet drops do not incur 
any penalty cost, the floating-queue algorithm achieves: 

/( p Q) = /( Dpp ) < /(°p‘) + 0 ( 1 /v) 

Per-hop delay < 0(B/(1 — e~ B )) = 0(B) 

Average drops < 0(e~ B ). 

It can be shown that the transient time of the drift-plus-penalty 
algorithm is 0(V), so parameter V cannot be set to infinity. 

Drop-Dependent Cost: 

In this case, packet drops affect the objective cost. Such cost 
can be the amount of admitted packets. Let k < oo be a 
maximum penalty cost per one unit of packet drop. Then we 
have the following result. 

Theorem 3: Suppose B > 2(c)( max ) +D). When V > 0 and 
k is a maximum penalty cost per one unit of packet drop, the 
floating-queue algorithm achieves: 

/( FQ) < / (opt) + 0(l/V) + 0(e~ B ) 

Per-hop delay < 0(B/( 1 — e -B )) = 0(B) 

Average drops < 0(e~ B ) 

Proof: Recall that f(t) is a cost incurred at time t under 
the drift-plus-penalty policy. At each time t, we have 

N 

/ (FQ) (t) < f(t) + K^d n (t). 

n= 1 

Summing from to to to + T — 1, dividing by T, and taking an 
expectation gives 


, to+T—1 to+T — 1 

- e E |/ (FQ) o)] < 7p E E t/w] 


t=to 


t=to 


K 

T 


to+T-l N 


E E E ^w]- 

t - t Q Tt - l 

Taking a limit as T approaches infinity gives 

to+T-l 

T—!-oo T E E [/ (FQ) (i) 

t=to 

< f( DPP) + KNS (max) c * e -{B/2-S^-D]/K 

</(°pt) + 0(l/v) + 0(e~ B ) 
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Fig. 7. Line network 




Fig. 8. Results of power minimization problem with V = 200 
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Fig. 9. Results of throughput maximization problem with V = 200 


VI. Simulation 

A line network with 4 queues, shown in Figure [7] is 
simulated in two scenarios. The common network configu¬ 
ration is as follows. In each time slot, an exogenous packet 
arrives with probability 0.92. Transmission (t) is orthog¬ 
onal and depends on channel state that is “good” with 
probability 0.9 and “bad” with probability 0.1 for (i. j) £ 
{(1,2), (2,3), (3,4), (4,0)}. 

A. Power Minimization 

In this scenario, all exogenous arrivals are admitted. When 
channel state is “good”, one packet is transmitted using 1 unit 
of power; otherwise 2 units of power are used. The goal is to 
stabilize this network while minimizing the power usage. Note 
that the optimal average minimum power is 1 x 0.9+2 x 0.02 = 
0.94 per hop, and the average total power is 3.76. 

Simulation results of this scenario are shown Figure [8] The 
time average power expenditure is 3.761 for all buffer sizes 

B. In Figure [8] the average delay increases linearly with the 
buffer size, and the average drops decrease exponentially with 
the buffer size. This result confirms the bounds in Theorem[2] 

B. Throughput Maximization 

In this scenario, a network decides to admit random exoge¬ 
nous arrival in each time slot. The goal is to maximize the 
time-average end-to-end throughput, which are real packets. 
Packet drops reduce the value of this objective function. 
Transmission p,ij(t) = 1 is possible if its channel state is 
“good”; otherwise the transmission is not allowed. Note that 
the maximum admission rate is 0.9, because of the limitation 
of the average transmission rate. Figure [9] shows the simulation 
results of this scenario, which comply with the bounds in 
Theorem [3] 

VII. Conclusion 

We propose the general floating-queue algorithm that allows 
the stochastic network optimization framework to operate with 
finite buffers. When the buffer size at each queue is B, we 


prove the proposed algorithm achieves within ()(e of 
optimal utility, while the average per-hop delay is 0(7?). The 
finiteness incurs 0(e~ B ) drops, decreasing exponentially. We 
confirm the theoretical results with simulations. 
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Appendix 

To prove case 2 of Lemma [4] we first show that, for 
any sample path {0„(£)}J_ T whose Qlit) is non-decreasing, 
there is a simple one that preserves the queue dynamics and 
the amount of admitted real arrivals under the floating-queue 
algorithm and the lower-bound policy. This simple sample path 
is then used to simplify the later proofs. 

Lemma 9: Given sample path {0 n (£)}J_ satisfying all 
dynamics in Section ITV1 where 0 n (£) = 

(< (9 > (9, < (9. a n (9. &n(9 - b l (9 > Qn (9, Qn (9) 

with Qlit) < Qlit + 1) for f e (r,...,r'}, a new sample 
path j(-)„(£) | where 0 n (£) = 

(< (9 - (9. < (9, a ll (9. f£(9 4n(9. Qn (9, Q f n (tj) 

with blft.) = min [Qlit), blit)] preserves all dynamics in 
Section [IV] 

Proof: From the dynamic of fake queue ( 1 1 9k if if) is 
the actual number of fake packets that is served at time 
£ under the floating-queue algorithm, i.e., Qlit + 1) = 
max [Qlit) - blit), 0] + a£ (t) = Q f n {t) - b f n {t) + a£ (t). 
Therefore, the dynamic of fake queue ( 1 1 9| i in the new sample 
path is still valid. The dynamic of real queue (TT8l) is valid, 
because the real part is not modified. 

■ 

The implication of the Lemma [9] is that, instead of an 
original sample path, we can consider its alternate version, 
in which fake services do not excess the amount of fake 

backlogs. Both sample paths shares the same admitted real 

/ . 

arrivals a r n (£) under the floating-queue algorithm. Further, 
both sample paths shares the same the admitted real arrivals 
a r n {i) under the lower-bound policy, because Q n {t) and a^(£) 
of both sample paths are identical. Notice that we need to 
preserve all dynamics in Section [IV] in order to use those 
dynamics and properties in Section [IV] and Section IV-A1 when 
an alternate sample path is considered. 

It is possible to simplify this alternate sample path. 

Lemma 10: Suppose B > 2dh" ax ). Then given sample path 
{0n(9K= T satisfying dynamics in SectionHVl where 0„(£) = 

(< (9 -««(9 - a n (9 > a ll (9. K (9 - b l (9. Qn (9: Ql ( 9 ) 


with Qlit) < Qlit + 1) and bfit) < Qlit) for t £ 
{t, .. ., t'}, a new sample path |0„(t)| where 0„(t) = 

(«n (9 - «n (9. a n (9, (9 - b n (9» b l (9»(9 > Ql (*)) 

with 

i) bl(t) = 0, 

ii) «n(9 = «n(9 - fe n(9, and 

iii) (9 = «n(9 + K(9 - (9) 

preserves all dynamics in Section [IV] 

Proof: For the given sample path {0„(f)}j_ T , fake 
backlogs are non-decreasing and Ql(t) < Ql(t + 1) for 
t € {r,..., t'}. Since bif f) < Qlit), it follows from ([19] ) that 
Ql(t- 1-1) = Qlit) —blit)+al it). From the non-decreasing, 
we have 

bl{t) < all{t). ( 24 ) 

From (fT3l ) and (IT4b . if blit) > 0, then Q r n (t) < b n (t) < <5( max ) 
and d n (t) = 0, since B > 2^( max ^. From ( fTT] ). it follows 
that al it) = alft). Using (l24l >. we have bl(t) < afit) 
and alft) — blft) > 0. Therefore, the new sample path with 
al(t) = al{t) — bfit) and blft) = 0 preserves the fake queue 
dynamic ( IT9l . i.e.. 


Qlit + i) 


= max [Ql(t) - blit), 0] + a£(f) + d n if) 

= Qlit ) - b lit) + alit) + «(9 - a r ^(t)) 
= Qlit) - bfit) + a f n it) + «(f) - all (9) 


= max 


Qlit) - blit), 0 


+ all it) 


where all it) = alit) + (a^(£) — a^(9)- The dynamic of real 
queue ( fl~8] > is valid, because the real part is not modified. ■ 

The implication of Lemma 0 and Lemma [TO] is as fol¬ 
lows. Given an original sample path {0n(9}t= T with non¬ 
decreasing fake backlogs, we can consider an alternate simple 

sample path , in which all dynamics and prop¬ 

erties of the floating-queue algorithm and the lower-bound 
policy still holds. Technically, the fake-backlog dynamic (THU ) 
becomes simpler, i.e., Qlit + 1) = Qlit) + alit) for all 
t £ {t, ...,t'} because bl(t) = 0. Also, the amount of 
admitted real arrivals under both float-queue algorithm and 
lower-bound policy are not changed. These simplify the later 
proofs. Before the main lemma, several properties are proved. 

Lemma 11: Suppose B > 2(5^ max i. Given any sample 
path {0ra(9}tLt,T satisfying all dynamics in Section [IV] 
for any t 0 and positive integer T, the following holds for 
k £ {1, • ■ ■, K)T)}, j £ {1,..., J(fc)}, tf :i - 1 > £ 0 , and 
* K(T),J{K(T )) + 1 < t 0 + T: 

• Qnitkj) = ^ 1), 

. afftfj- 1)=0, 

• Qlit~ kj -l)<lll*-B/2 + 5^\ 

• Qlit+, + 1) <^*-8/2 + 6^. 

Proof: For the first property, all real backlogs Q' n (tf , - 1) 
must be served at time £^ - — 1, in order to have bl(ff ■ — 1) > 0 
and Qlitfj - 1) > Qlftfj), so Qfitf^ - 1) - bfitf^ - 






1) = 0. From Lemma [2] Q f n itf.) < Q(, (tfj - 1) implies 
a r n (t) = a r n ( t ). Thus, it follows from (fl8] > that Q r n {tf ■) = 
0 + • — 1), which proves the first property. 

For the second property. Let b n (tf ■ — 1) = 


QnifkJ 7 - 1) 


k,j 


We 


shows 


that 


Q n (t k j — 1) ^ B n by considering the dynamic of the 


standard queue (fl2l >: 


Qn/tkj 1 ) — Qn(tk,j ) b n{^k,j 

= Qni^k,j) + Qni/kJ ) + b n(t k j — 1 ) — a n (t k j — 1 ) 

= On(*fcJ ~ !) + Q^kj) + ~ b n(.t kJ - 1) - a n {t k j - 1) 

< 7^* - B/2 + 6^ (/ B n . 


The last equality uses the first property. The last inequality 
uses the facts that a r n (t k] -1) < a n (t k ■ - 1) and Q f n [t k ■) < 
lX*-B/ 2. 

The third property can be proved from ( fl9l > as follows: 


Ql(t k ,j) = max Qn(h,j -!) - hlihj - i )’ 0 

— QrJtkJ — 1) — b n(t k j — 1), 

and 


J'u- 


( b kj 


1) = Qn(t) +a£{t) for all t G ... ,f^}. Summing 

from t = t k j to yields 

‘w 

+ 1) = + E “nW 

= Q™(^)+ E [“£(*) + d «( t )]> 

t=t k.j 

where the last equality uses 03. Rearranging terms gives 

E = Qltttj +!) - Qlitkj) - E a «W- 


Since Q f n {t k j ) is a local minimum and Q f n {t k j ) < Ql(t kj - 
1), Lemma |2] implies that d n (t k . — 1) = 0. From the above 
equation, we have that 
1) . + 


fc.j 




E d n(t) = Q f n (4 tj + l)-Q f n (t kJ )- E “nW- <^ 26 ) 


u k,j 


Qlitkj- 1) < + < 7 r-5/2 + <5( max ). 

The last inequality uses the fact that Ql(t k ■) < 7^* — B/2. 

Similarly, the last property can be proved from 03 as 
follows: 


Qlit-tj + 1) = max Q{(f+.) - b s n (t+ tj ), 0 
<7^*-B/2 + ^ max ). 


(t 


kj > 


The last inequality uses the fact that Ql(t k ■) < 7^/* — B/2. ■ 
We now quantify packet drops under the floating queue 
algorithm via a modified sample path. The sample paths 
between t k - and t k j for some k and j can be either “non¬ 
decreasing (at the end)” type or “decreasing (at the end)” 
type. This depends on the non-decreasing or decreasing of 
fake backlogs at the end of the sample path. For example, 
in Figure [6] a sample path between if 2 and /,J. 2 is the non¬ 
decreasing type, and a sample path between /:/ , and //, is 
the decreasing type. We prove each type separately. 

Lemma 12: When the buffer size of real queue n is 
B > 25^ x \ given a non-decreasing-type sample path 

{©n(i)}E- .-1 with b l (*) = 0 for a11 1 e { l k,P ■ ■ 

the following holds 

k,j L k,j 

E a nit)> E “nW’ (25) 


Proof: We first consider drops under the floating-queue 
algorithm. From (fl9] i with &£(i) = 0, it follows that Ql(t + 


Now, drops under the lower-bound policy is considered. We 

£+ 

suppose that k f_ d n (t ) > 0; otherwise, there is no drop 

under the floating-queue algorithm, and 


E a n (*) = E - E “nW- 


From Lemma [TTl we know that d r n (t k ■ — 1) = 0. Therefore, 
we suppose that Q n (t) > j//* — B/2 + for some t G 
| tf j,.... otherwise, the lower-bound policy drops all 

real arrivals, and 

k,j L k,j 

E «n( i ) = °< E a n0)- 

When the above assumptions are imposed, we let 

tlj = arginf t£{t - t +, } [Q n {t) > - B/2 + ^}, 

be the first time that Q n (t) is at least 7^"* — B/2 + ( i >( max ). This 
is illustrated in Figure [6] Notice that t* k 7 G ? ,.... t//^ | 
by the later assumption. 

From (fl8l> and (fT9l > with bl(t) = 0, it holds for t G 
{ b kji - • ■ > t k j — l} that 

Quit + 1) = Qh(t + 1) + Ql(t + 1) 

= Q r n(t ) - b r n (t) + <'(f) + Ql(t ) + all(t) 

= Quit) - (£(f) + <'(f) + a({t). 








Summing the above equation from t = t k ■ to t* k ? — 1 yields 

tfcj-t 

Qn(ifc,j) = Qn(tk,j) + E [“&«(*) + On 0) + “»'(*) 

t=t fcd 

ifcj-t 

= < 9n(*fe,j)+ E [ _& n(*) +°nW + «£(*)]> 


and 

tij- 1 n.j-i 

E a nW = Qn(tk,j) ~ Qn(tk,j) + E [ 6 n(*) “ a n(i)] 

> Qn(t k j) — Qn{t k j) — El a n(^) 

>7^-B/2 + 5(”“)-Q n (t^.)- E a n(*) 

t=t k,j 

tk.j- 1 

>Qn( t k,j + X ) ~ a n(Ej - !) + <2n0fcj) - E a «(^' 

t=t k,j 

The last steps uses Lemma fTTI and the facts that Q n {t k j) = 
Q r n(t kJ ) + QUt 

From the definition of t k • , we know that Q n (t) < 7 ^* — 
B/2 + S^- max ^ and all real arrivals under the lower-bound policy 
are dropped, a r n {t) = d n (t), for all t G j t kj ,... ,t k j - lj. 
The third property in Lemma QT| also implies that a r n {t k j — 
1) = d n (t k j — 1). Therefore, the above inequality yields 

E = E 

£=£“ -1 £=£“ -1 

> + !) " - E “"W 

t=t k,i 

> Qi(t+ • + 1) - Qi(t kt ■) - E «£(*) 


‘'fc.J 


"fc.J 


— ^ ] d n (t ), 


where the last step uses (1261) . Then we have 




‘'fc.j 


y: 4(t) > dn(o* 

t=t7 -1 t=t7 -1 

Applying (fl6l> and ( 122b proves the lemma. Note that the 
equality holds in some situations when the both algorithms 
do not drop any real packets. ■ 


Lemma 13: When the buffer size of real queue n is B > 
2<5( max ) , given a decreasing-type sample path {0 n (£)} 

with bl{t) = 0 for all t G ..., t] z j - l} and Q 1,71 7 > 

Qni^kj + 1), the following holds 


L k,j 


°k,j 


E °n (*) ^ E 


Proof: We construct 


a new sample path |@ ra (f)| 


k,j 

t=t 


k,j 


where 0 n (t) = 0 n (f) for all t G ..., - lj. 

Applying Lemma fl2l on the new sample path implies that 


E «;'(*)> E *»(*)• 


Since Q f n { + 1) < Lemma |2] implies that 


k,j 1 


= a n(H,j)- ThiS meallS that ^ «n(E)’ 

Using this and the above relation proves the lemma. ■ 

Finally, the second case of Lemma [4] can be proved. 
Lemma 14: When the buffer size of real queue n G Af 

is B > 25( max ), given any sample path {0„(t)} _ 7 
satisfying all dynamics in Section [TV] it holds that 


u k,j 


L k,j 


E a n (t) > E “«(*)• 


Proof: We construct a new sample path and apply a 
previous lemma, depending on the type of the original path. 


Let 0 


{©, 


‘k,j 


be a new sample path with 0„ (f fc . — 


1) — ©ra(ffc j !)• 

When Ql(t ki + 1) > Ql(t k ■) (non-decreasing type), let 


{ 


0 r 




be a result of applying Lemma [9] and Lemma 


"k,j 


UOl on {(-)■„.( t) } k f 3 - ■ Then, applying Lemma fl2l on the new 
sample path proves the lemma. 

When Ql,itX kJ + 1) < Q f n (tt,j ) (decreasing type), let 

\ 0 n (f) r ' be a result of applying Lemma 0 and Lemma 

union {0n(f))E- 1 ’ and let 0n(ffc ; j) = ©n(£fc ;j ). Then, 
applying Lemma|T2]oii the new sample path proves the lemma. 





